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\^ , Abstract 

^ . 
■4-J Although the classical LQR design method has been very successful in real world engineering 

designs, in some cases, the classical design method needs modifications because of the saturation in 

actuators. This modified problem is sometimes called the constrained LQR design. For discrete sys- 

,^^ tems, the constrained LQR design problem is equivalent to a convex quadratic programming problem 

l/^ ' with box constraints. We will show that the interior-point method is very efficient for this problem 

\^ ' because an initial interior point is available, a condition which is not true for general convex quadratic 

• , programming problem. We will devise an effective and efficient algorithm for the constrained LQR 

(^ , design problem using the special structure of the box constraints and a recently introduced arc-search 

m ■ 

technique for the interior-point algorithm. We will prove that the algorithm is polynomial and has the 

best-known complexity bound 0(ynlog(l/e)) for the convex quadratic programming. The proposed 
- 1 — \ 
^^ ' algorithm is implemented in MATLAB. An example for the constrained LQR design is provided to 

jrt , show the effectiveness and efficiency of the design method. The proposed algorithm can easily be 

used for model predictive control. 

Keywords: Constrained LQR, arc-search, convex quadratic programming, polynomial algorithm. 



1 Introduction 

The LQR design of linear control system theory has been proven to be one of the most effective ways to 
design real world control systems because, in many cases, the linear system model has adequate accuracy 
and fidelity to describe the real world systems, the design method has manageable complexity, and is 
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better understood and more mature than the design methods in nonlinear system theory. For example, 
a beautiful application that demonstrated all these nice features is described in 18 . One of the major 
obstacles in some applications for the LQR design or, in general, the classical linear control system theory 
is that it does not consider the reality that the actuators have some saturation limits. This drawback 
in the linear control system theory has been addressed by many people and some of the latest results 
are presented in some recently published books such as [TT] [3S] . Two of the most attractive and related 
methods are the constrained LQR design [HHT] and the Model Predictive Control (MPC) design [T71I2]. 
both involve solving some convex quadratic programming. 

In |21| . a general framework of the constrained LQR problem is formulated. The problem can be 
represented as a standard convex quadratic programming with Nr + Nm variables, where N is the 
number of horizons, m is the number of control inputs, and r is the number of states. This problem is 
then solved by using two interior-point methods. The first method is similar to the one proposed in [3], 
the second method is similar to the one proposed in 13' . The analysis and computational experience 
demonstrate the same frustration that the interior-point algorithms with desirable theoretical properties 
(polynomial complexity) tend to be slow in computation, while little can be proven about the algorithms 
(such as Mehrotra Predictor Corrector algorithm [121 ED]) that perform much better in practice. 

MPC design is a very active research area which has many industry applications [16 and numerous 
theoretical investigations 1 . A very attractive and important method is proposed in 2, where a general 
frame work of model predictive control problem is formulated. To reduce the on-line computational 
burden, the off-line design strategy is emphasized. The problem is solved by a multi-parametric program in 
which some convex quadratic programming will be solved repeatedly. Further considerations on efficiency 
is discussed recently by many people, for example |19j . These papers focus on the off-line strategy but 
pays no attention to the development of the best convex quadratic programming method for the MPC 
design. 

In this paper, we will consider constrained LQR design problem in which the actuators have lower 
and upper bounds. This problem is slightly simpler than the problems considered in [211 [3J 119) but is 
still general enough for most real world problems. We will show, by using the state space equations, that 
the number of the variables of the constrained LQR problem described in |21| can easily be reduced from 
Nr + Nm to Nm and all equality constraints can be removed. This means that the reduced problem 
is not only much smaller but also has a special structure which is called convex quadratic programming 
subject to box constraints. We will solve the reduced problem by an interior-point algorithm that searches 
the optimizer along an arc that approximates the central path. This algorithm is similar to a recently 



developed algorithm [53] but it is especially designed for convex quadratic programming subject only to 
the box constraints and it is more efficient than the algorithm in |24| because of two improvements due 
to the special structure: (1) the enlarged search neighborhood, and (2) an explicit initial interior point 
(finding an initial interior point may be a major obstacle for feasible interior-point methods). We will 
show that this algorithm has both desirable theoretical properties (polynomial complexity) and superior 
performance in computation because of the above mentioned improvements. Although the idea of the 
proof of polynomiality is similar to that used in [21J , we provided the proof in the appendix for several 
reasons: (1) it also shows the enlarged search neighborhood comparing to the linearly constrained convex 
quadratic programming, (2) some special cares are needed for box constraints, and (3) the completeness. 
We have implemented the algorithm in MATLAB. We will demonstrate by some numerical LQR design 
example that the proposed constrained LQR design is very effective and efficient. 

The remainder of the paper is organized as follows. Section 2 introduces notations and some technical 
lemmas that will be used in the rest of the paper. Section 3 discusses the constrained LQR design and 
convex quadratic programming with box constraints. Section 4 describes the central path of quadratic 
programming with box constraints. Section 5 proposes an arc-search algorithm for the convex quadratic 
programming with box constraints. Section 6 gives convergence analysis. Section 7 addresses implemen- 
tation issues. Section 8 presents some LQR design example. Section 9 summarizes the conclusions. Some 
technical proofs are in the appendix to enhance the readability of the paper. 

2 Some Notations and Technical Lemmas 

Throughout the paper, we will use notations adopted in |24j . We denote n-dimensional vector space by 
R", n X m-dimensional matrix space by R"^™, Hadamard (element-wise) product of two vectors y G R" 
and A G R" by y o A, the ith component of y by yi, element- wise division of the two vectors by ^ if 
minjAil > 0, the Euclidean norm of y by ||y||, the identity matrix of any dimension by /, the vector of 
all ones with appropriate dimension by e, element- wise absolute value vector by \y\ — [\yi\, . . . ,\yn\f^ ■ 
To simplify the notation for block column vectors, we will denote, for example, [y'^,X"^]'^ by (y. A). For 
vectors x G R", y G R", z G R", A G R", and 7 G R", we will use capital letters X, Y, Z, A, and 
F for some related diagonal matrices whose diagonal elements are the components of the corresponding 
vectors. For example, we will use A = diag(A) and F == diag(7) for the diagonal matrices. For a matrix 
H G R"^", we use H > ii H is positive semidcfinitc, and i/ > if if is positive definite. Finally, we 
define an initial point of any algorithm by a;", the point after the fcth iteration by x'' . 



We will also use some technical lemmas which are independent of the problem. The first two simple 
lemmas are given in |22] . 

2 

Lemma 2.1 Let p > 0, q > 0, and r > be some constants. If p + q < r, then pq < ^ . 
Lemma 2.2 For a G [0, f ], 

sin(a) > sin^(a) = f — cos^(a) > f — cos(q;). 

The following Lemma is proved in |14| . 
Lemma 2.3 Let u, v, and w be real vectors of same size satisfying u + v — w and u^v > 0. Then, 

2\\u\\ ■ \\v\\ < \\ur + \\vr < M' + Wvf + 2u^v = ||h + vf = Wwf. (1) 

The next technical lemma is from [3D1 page 88]. 
Lemma 2.4 Let u and v be the vectors of the same dimension, and u^v > 0. Then 

\\U0V\\ < 2"5||u-|-„||2_ 

We will use the famous Cardano's formula which can be found in |15) . 
Lemma 2.5 Let p and q be the real numbers that are related to the following cubic algebra equation 

x^ +px + q = Q. 
If 



-2/ V3 
then the cubic equation has one real root that is given by 
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(I) 
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For quartic polynomials, the roots can also be represented by formulae, we do not list all the possible 
cases and solutions but refer to 10 for the detailed discussion. The last technical lemma in this section 
is as follows. 

Lemma 2.6 Let u and v be the n-dimensional vectors. Then 



u o V (u v) e 

n ^ ' 



< 



UO V 



Proof: Omitted. 

3 Constrained LQR and Convex QP with Box Constraints 

We will consider the following constrained LQR (or MFC) design problem. Let x e R^ be the system 
state, and u € R™ be the control vector. The discrete linear time-invariant system is given by 

x^+i =.4x, +Bu^, (2) 

while fulfilling the constraints 

- e < Us < e, (3) 

where s = t, . . . ,t + N — 1. Let P, Q, and R be positive definite matrices. The design is to optimize the 
following cost function 

1 1^"' 

J= min -x7+ArPxt+Ar + - V [x7_^fcQXf+fc+u7+fci?U4+fc] (4) 

under the system dynamics equality constraints ^ and control saturation inequality constraints ([3]). 
Given current state xt, this LQR (or MFC) design problem is a typical convex quadratic programming 
problems with Nr + Nm variables xj+i, • • • , Xf+^Vi Uj, • • • , Uf_|_7v-i- Though this problem can be directly 
solved as suggested by |31 121) , it can be significantly reduced to an equivalent but much smaller convex 
quadratic programming problem subject only to box constraints. Denote 

A'' ^ A---A ■.= Ak 
product of k A 

with Aq ~ I. Since 

fc-i fc-i 

xt+k = Axt+k-i + But+k-i = A^xt + ^ A^But+k-j-i = ^fex* + ^ AjBut+k-j-i, (5) 

3=0 i=0 



^ can be rewritten as 



N-l 



N-1 



J = min 



^.. fc-1 \^ / ^"' \ l'^"' 



AT-l 



fc=l 



/£ = 



Notice that Xt is a constant vector, Aj, P, Q, and i? are constant matrices, the cost function ^ can be 
reduced to 



Denote 



N-l 



N-1 



Jo = min 



J=0 



'N-l 



I J2 AjBxit+N-j-i \ P\Y1 AjBut+N-j-i 
{ANVitfP ( ^ AjBut+N-j-i 



' k-l 



j=0 



k=l \]=0 

N-l 

Y, {{Ak^tfQ{Y.^iBut+k-j-i 



' k-l 



k=l 
N-l 



yj=0 
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fc=0 



fe-1 



1=0 ' ;r ' 



Ut 



Uf+fc-1 



4ikVk, 



(7) 



(8) 



Qfc 



^J( 



i?jV — 



R ■■■ 



••• R 



N diagonal matrices 



(9) 



(10) 



and 



AlQ(l)k 



(11) 



where are zero matrices with appropriate dimensions. The constrained LQR (or MPC) design is reduced 
further to 

Jo = min -vjf cj)^Pcj)N + T^ Qfc + i?w "at + x^ A^P^w + V" 5"^ vn 

\ k=l / \ k=l J 

s.t. — e < Wat < e. (12) 

This is a convex quadratic programming problem with Nm variables and 2Nm box constraints, a much 
smaUer and simpler problem than the original one. Let n = Nm, 

X = VN, (13) 



N-l 



H = 4'JlP^N + ^ Qfe + i?Ar , (14) 



fc=l 



cT=xn ATp0^+^5fcj. 



(15) 



The LQR (or MPC) design problem can be written in a standard form of convex quadratic problem with 
box constraints: 

(QP) min ^x'^Hx + c^x, subject to — e < x < e, (16) 

where < H E fj^nx" is a positive definite matrix, c e R" is given, and x G R" is the control vector to 
be optimized. The remaining discussion of this paper is focused on the solution to the convex quadratic 
programming problem with box constraints described by (J16p . 



4 Central Path of Convex QP with Box Constraints 

It is well known that x is an optimal solution of (jl6|) if and only if x, A, and 7 meet the following KKT 
conditions 

-X + j~ Hx^c, (17a) 

-e<x<e, (17b) 

(A,7)>0, (17c) 

Aj(ei - Xi) = 0, ji{ei + Xi) = 0, i = l,...,n. (17d) 



Denote y = e — x>0, z = e + x>0. The KKT condition can be rewritten as 

iJa; + c + A-7 = 0, (18a) 

X + y — e, X — z = — e, (18b) 

(y,z,A,7)>0, (18c) 

Aj2/i = 0, 7iZi = 0, i^l,...,n. (18d) 

For the convex (QP) problem, the KKT condition is also sufficient for x to be a global optimal solution. 
Denote the feasible set J^ as a collection of all points that meet the constraints (|18a|) . (J18bp . (|18cp 



T = {{x, y, z, A, 7) : Hx + c + A — 7 = 0, (y, z, A, 7) > 0, x + y = e, x — z = — e}, (19) 

and the strictly feasible set F° as a collection of all points that meet the constraints (|18ap . (|18bp . and 
are strictly positive in (I18cp 



J"° = {(x, y, z, A, 7) : Hx + c + A — 7 = 0, (y, z, A, 7) > 0, a; + j/ = e, x — z = — e}. (20) 

Similar to the linear programming, we define the central path C G J-° C J^, as a curve in finite 
dimensional space parameterized by a scalar t > as follows. For each interior point {x, y, z. A, 7) e T° 
on the central path, there is a t > such that 

Hx + c + X-'y = 0, (21a) 

X + y — e, X — z = — e, (21b) 

(y,z,A,7)>0, (21c) 

hyi=T, j^Zi = T, i = l,...,n. (21d) 

Therefore, the central path is an arc that is parameterized as a function of r and is denoted as 

C = {{x{T),y{T),z{T), A(t), 7(t)) : t > 0}. (22) 

As T — > 0, the moving point (x(r), j/(r), z(t), A(t),7(t)) on the central path represented by (|2ip ap- 
proaches the solution of (QP) represented by p^ . Throughout the paper, we make the following as- 
sumption. 

Assumption: 

1. J^° is not empty. 



Assumption 1 implies the existence of a central path. This assumption is always true for the LQR 
problem, and we will provide an explicit initial interior point in Section 7. 

Let 1 > 6* > 0, denote p — {y,z), w = (A, 7), and the duality gap 

X'^y + j'^z p^uj 

We define a set of neighborhood of the central path as 

MM = {{^. y. z. A, 7) e ^° : lb o c^ - ^e|| < 0^, } c T". (24) 

As we reduce the duality gap to zero, the neighborhood of A/'2(6') will be a neighborhood of the central 
path that approaches the optimizer(s) of the QP problem, therefore, all points inside A/2(0) will approach 
the optimizer(s) of the QP problem. For (a;,y, z, A,7) G M^ifi), since (1 — Q)p < ujiPi < (1 + 6*)/!, where 
uJi are either Ai or 7^, and pi are either yi or z^, we have 

ujiPi ma.XiU!iPi miiiiUJiPi uJiPi 

— n — ^ M ^ 7 — ^ 77- (25) 
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5 An Arc-search Algorithm for Convex QP with Box Constraints 

The idea of arc-search proposed in this paper is very simple. The algorithm starts from a feasible point 
in J\f2 (0) close to the central path, constructs an arc that passes through the point and approximates the 
central path, searches along the arc to a new point in a larger area Mi^O) that reduces the duality gap 
p'^uj and meets (|21b .). (|21b ). and (PTb ). The process is repeated by finding a better point close to the 
central path or on the central path in Mid) that simultaneously meets (l?Tk ). (121b ). and (pTb ). 

Following the idea used in [21], we will use an ellipse £" [B] in an appropriate dimensional space to 
approximate the central path C described by (|2T|) . where 



£ — {{x{a),y{a),z{a),X{a),j{a)) : (x(a), j/(a), z(a), A(a),7(a)) = acos(a) + 6sin(a) + c}, (26) 

a e R^" and b e R^" are the axes of the ellipse, c e R^" is the center of the ellipse. Given a point 
(a;, y,z,A,7) — {x{aQ),y{ao), z{ao), X{ao),j{ao)) G £ which is close to or on the central path, a, 6, c 
are functions of a, (x. A, 7, y, z), (i, y, i. A, 7), and [x, y, z, A, 7), where (i, y, i. A, 7) and (x, y, z. A, 7) are 
defined as 



The first rows of ^7^ and 



HO / -/ 
/ / 
/ -/ 

A y 
r z 

HO I -I 
I I 
10-10 

A r 
r z 

are equivalent to 





X 









ij 









Z 


= 







A 




Aoy 




7 




70Z 



(27) 



X 

y 

z 
X 

7 







-27 O Z 



(28) 



Hi = 7 — A, Hx = 7 — A. 



Tlic next 2 rows of (P7)) and ([^5]) are equivalent to 



X = — y, X = z, a; = — y, x = z. 



The last 2 rows of (|27|) and (|28p are equivalent to 



pocj+pooj =pow, 



pouj~\-pou) — ~2p o cj. 



(29) 



(30) 



(31) 



(32) 



It has been shown in [22] that one can avoid the calculation of a, 6, and c in the expression of the 
ellipse. The following formulas are used instead. 

Theorem 5.1 Let (^(q;), y{Q), z{a)^ A(a), 7(a)) be an arc defined by i26\) passing through a point (x, y, z, A, 7) G 
£ , and its first and second derivatives at {x, y, z, A, 7) be {x, y, z, A, 7) and {x, ij, z, A, 7) which are defined 
by {21^ and ^2^) . Then an ellipse approximation of the central path is given by 



x{a) = X — isin(a) + x{l — cos(q;)), 

y{a) = y -ysm{a) + y{l - cos(a)), 
z{a) = z — isin(a) + z(l — cos(a)), 
X{a) — X — Asin(Q;) + A(l — cos(a)), 



(33) 

(34) 
(35) 
(36) 
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7(a) = 7 ^ 7sin(a) + 7(1 — cos(a)). (37) 



We will also use a compact format for p{a) = {y{a), z{a)) and uj(a) ~ (A(a),7(a)), which are given 

by 

p{a) = p — psin{a) +p{l — cos{a)), (38) 

w(a) = u; — cJsin(Q;) + a}(l — cos(a)). (39) 

We denote the duality gap at point {x{a),p{a),uj{a)) as 

^ X{a)'^y{a)+j{a)'^z{a) ^ p{a)'^uj{a) 
^^ ' 2n 2n ' ^ ' 

Assuming (y, z,A,7) > 0, one can easily see that if -, |, j, -, -, |, j, - are bounded (we will show 
that this is true), and if a is small enough, then y{a) > 0, z{a) > 0, A(a) > 0, and 7(a) > 0. We will 
also show that searching along this ellipse will reduce the duality gap, i.e., /i(a) < fi. 

Lemma 5.1 Let (x,y, z, X,^) be a strictly feasible point of (QP), (i,y, i,A,7) and (x,j/,z, A,7) meet 
P7P and figj, (a;(a),z/(a),z(Q!), A(a),7(a)) be calculated using l^, ^B^, l^, (^, and ^^, then the 
following conditions hold. 

x{a) + y{a) ~ e, x{a) — z{a) = — e, Hx{a) + c + A(q;) + 7(0;) = 0. 



Proof: Since (a;, y, z, A,7) is a strictly feasible point, the result follows from direct calculation by using 
(120]) . ((271) . (EH), and Theorem O | 



Lemma 5.2 Let (i;,p, w) be defined by {27^ , {x,p,uj) be defined by i28\) . and H be positive definite matrix. 
Then the following relations hold. 

p^ilj^x'^{'j-X)^x^Hi>0, (41) 

The equality holds if and only if \\x\\ = . 

p^Ld^x^{j-X)^x^Hi>0, (42) 

The equality holds if and only if \\x\\ = 0. 

p^co = i;'^(7 - A) = i'^(7 - A) == p^uj = x^Hi. (43) 
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< (x^ij - A) + i'^(7 - A)) sin(a)(l - cos(a)) 

< {x^Hx){l - cos{a)f + {x^Hi)sm^{a). 



(44) 



-{i'^Hi) sin^{a) - [x^ H£){1 - cos{a)Y 

< {x^{j - A) + x^{^ - A)) sin(a)(l - cos(a)) 

< (x'^Hx) sin^ia) + {x'^Hi){l - cos{a)f. 

For a = ^, ^44^ <^^d, ^5\ l reduce to 

- {x^Hx + x^Hi) < [i^Hx + x^ Hi) < x^Hx + x^ Hi. 



(45) 



(46) 



Proof: See Appendix A. 

Using Lemmas Ea EH and EH we can show that ^ := (i,f), % := (|,i), f := (f , f ) 
^ :— ij,-'-) are all bounded as claimed in the following two Lemmas. 

Lemma 5.3 Let (x,p,uj) — {x,y,z,X,j) £ M2{d) and {x^p,uj) = {x,y, ZjXjj) meet [S^ . Then, 

2 2n 

< r. 



and 



p 


2 


w 




+ 


— 


V 




UJ 


V '■ 


2 w 


2 




— 


< 


V 


w 





1-1 



0< 



■p^LO 1 + 



< 



^ - 1- 



-n := Sin. 



(47) 

(48) 
(49) 



Proof: See Appendix A. 

Lemma 5.4 Let {x,p,uj) — {x,y,z,X,j) G Af2{0), (i,2/,i,A,7) and (x,y,z,A,7) meet |^7| ) and 

Then 

2 4(1 + 6l)n2 



p 


2 


UJ 




+ 


— 


p 




U! 


p ' 


'- W 


2 




— 


<- 


p 


CJ 





(1-0)3 ' 



2 /2(l + 6i)n2^^ 



P^Cj 2(1 + 61)2 



•T •• 



M " (1 

^ (2n(l + e))i ^ 3 

- (1-0)2 • "3" ' 



••T • 



^(2n(l + 0))5 3 

- (1-0)2 • "3'' ■ 



(50) 
(51) 
(52) 
(53) 
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Proof: See Appendix A. 

Using the bounds established in Lemmas 15.21 15.31 15.41 and 12.21 we can obtain the lower bound and 
upper bound for //(a). 

Lemma 5.5 Let {x,p,uj) = {x,y, z,X,j) £ Af2{0), {x,y, z,\,^) and (x, y, z, A,7) meet \2T^ and 
Letx{a), y{a), z{a), \{a), and -/{a) be defined by ^, (Ml, (3^, fS^j, and (3^. Then, 



nil - sin(a)) x^Hx ((1 - cos(a))^ + sin^(a)) 

2n 

<^(q) = ^(1 - sin(a)) H ( ^'''(7 - A) - ±'^(7 - A) ) (1 - cos(a))^ 

2n V / 

(x'^(^ -X)+x^('j- X)] sin(a)(l - cos(a)) 

2n V / 

<H(1 - sin(a)) + —i^Hi ((1 - cosfa))^ + sin^fa)) . (54) 

2n 



Proof: See Appendix A. 

To keep all the iterates of the algorithm inside the strictly feasible set, we need (p(a),a;(a)) > for 
all iterations. We will prove that this is guaranteed if /i(a) > holds. The following corollary states the 
condition for /i(a) > to hold. 

Corollary 5.1 // /i > 0, then for any fixed 6 e (0, 1), there is an a > depending on 9, such that for 
any sin(a) < sin(Q), /i(a) > 0. In particular, if ~ 0.19, sin(a) > 0.6158. 

Proof: From Lemmas 15.21 and 12.21 we have x^ Hx^ = ^^ {i ~ ^) — P^^ ^^d ((1 — cos(a))^ < sin (a). 
Therefore, from Lemmas 15.51 and 15.31 we have 



/i(a) > ^ I 1 — sin(Q;) — - — p^ioi sin^(a) + sin^(a) j j 

>^i(l- sin(a) - j-t-l(^sm^{a) + sin2(a)) j := ^ir{a). 

Since /i > 0, and r{a) is a monotonic decreasing function in [0, ^] with r(0) > 0, ?"(§) < 0, there is a 
unique real solution sin(a) € (0, 1) of r{a) — such that for all sin(a) < sin(a), r(a) > , or ^J.{a) > 0. 
It is easy to check that ii 9 = 0.19, sin(a) = 0.6158 is the solution of r{a) ~ 0. | 

Remark 5.1 Corollaru \5.1\ indicates that for any 9 e (0, 1), there is a positive a such that for a < a, 
p{q) > 0. Intuitively, to search in a wider region will generate a longer step. Therefore, the larger the 9 
is, the better. But to derive the convergence result, 9 < 0.22 is imposed in Lemma \5.9\ and 9 < 0.19 is 
imposed in Lemma \5.13i 
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To reduce the duality gap in an iteration, we need to have fJ.{a) < fi. For hnear programming, it is 
known |22| that /i(a) < fi for a € [0, a] with a — ^, and the larger the a in the interval is, the smaller 
the /i(a) will be. This claim is not true for the convex quadratic programming with box constraints and 
it needs to be modified as follows. 



Lemma 5.6 Let {x,p,Lu) — {x,y, z,X,j) e J\f2{0), {x,y, z,X,^) and {x,y,z,X,j) meet \21/^ and i fM) 
Let x(a), y{a), z{a), \{a), and 7(a) be defined by hSS^. {34% V35\). V3b\) . and JgTp . Then, there exists 



where 



2 ' y nji — ^ 



sin H5), if 



x' Hx 



> 1 




n/i 
^x"^ Hx 

such that for every a G [0, a], /i(a) < fj,. 



nfi 



' nfi 
^x^ Hx 



(55) 



Proof: See Appendix A. 

According to Theorem l5.ll Lemmas 15. 11 15. 3[ 15.41 and 15.61 if a is small enough, then (p(a),a;(a)) > 0, 
and /^(a) < /^, i.e., the search along the ellipse defined by Theorem 15.11 will generate a strictly feasible 
point with a smaller duality gap. Since (p, uj) > Q holds in all iterations, reducing the duality gap to 
zero means approaching the solution of the convex quadratic programming. We will apply a similar idea 
used in [131 123) . i.e., starting with an iterate in A/'2(0), searching along the approximated central path 
to reduce the duality gap and to keep the iterate in A/'2(20), and then making a correction to move the 
iterate back to J\f2{0). First, we will introduce the following notations. 



ao 



-Ofi < 0, 



ai = Oji > 0, 
a, = 20^^ = 20^^^^ = 20^ > 0, 



2n 



2n 



2n 



as. 



p o Lu + oj o p (p oj + u! p)e 

2n 



>0, 



04 



p o uj — uj o p (p ^ — Lo p)e 

2n 



poLJ — Ljop— ——(p u — u) p)e 
2n 



29 



T • 
p UJ 

~2n 



x^ Hi 

20—- > 0. 

2n - 
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We also define a quartic polynomial in terms of sin(Q;) as follows 



q{a) — 04 sin** (a) + a^ sin'^(a) + a2sin^(a) + oi sin(a) + oq = 0. 



(56) 



Since q{a) is a monotonia increasing function of a £ [0, ^], g(0) = —9^ < and <z(f ) = 02 + 03 + 04 > if 
i 7^ 0, the polynomial has exactly one positive root in [0, -j]. Moreover, since (j56p is a quartic equation, 
all the solutions are analytical and the computational cost is independent of the size of H and negligible 

m- 

Lemma 5.7 Let {x,p,uj) = {x,y,z,X,Lu) G M2{d), {x,y,z,X,uj) and {x,y,z, X,LJ) be calculated from 
|i?7[ ) and Ii28\) . Denote sin(Q!) be the only positive real solution of 156]) in [0,1]. Assume sin(a) < 
min{sin(Q;), sin(Q;)}, let {x{a),y{a), z{a), X{a),j{a)) and fi{a) be updated as follows 

(x{a),y{a),z{a), X{a), 7(a)) = (a;, y, z, A, 7) - (x, y, z, A, 7) sin(a) + {x, ij, z. A, 7)(1 - cos(q;)), (57) 

^i(a) = p(l - sin(a)) + -— ({p^u) - p^uj){l - cos(a))^ - (p^iv + p^uj) sin(a)(l - cos(a)) ) . (58) 

Then (x(a),2/(a),z(a),A(a),7(a)) eJV2{20). 

Proof: See Appendix A. 

Remark 5.2 It is worthwhile to note, by examining the proof of Lemma \5.T\ that sm{a) is selected for 
the proximity condition Iil03\) to hold, and sin(a) is selected for p{a) > 0, thereby assuring the positivity 
condition \104^ to hold. 

The lower bound of sin(Q;) is estimated in Corollary 15. II To estimate the lower bound of sin(a), we 
need the following lemma. 



Lemma 5.8 Let {x,p,uj) e J^2{6), {i,P,'^) and [x,p,ij) meet P7| ) and if^J. The 



p O LO 






po UJ 



< 



2(1 



ii~ 9y 



p O LO 



p O LO 



< 



< 



2\/2(l 



(i-ey 

2%/2(l + 6 



-n^fi, 



)2 3 
—n^p, 



)2 3 

— n2^. 



(59) 

(60) 
(61) 
(62) 
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Proof: See Appendix A. 

Lemma 5.9 Let 9 < 0.22. Then sin(a) > -^. 
— \ / — ^ 

Proof: See Appendix A. 



Corollary 15.11 Lemmas 15.71 and 15.91 prove the feasibility of searching optimizer along the ellipse. To 
move the iterate back to J\f2{6), we use the direction (Ax, Ay, Az, AA, A7) defined by 



H 








/ 


-J 


I 


/ 











I 





-/ 









/ 



Aa; 
Ay 

Az 
AA 

A7 



^{a)e — A(a) o y{a) 
fj,{a)e — 7(a) o z{a) 



A(a) Y{a) 
r(a) Z{a) 

and we update {x'''^^ , p''^^ , u'''^^) and fi''^^ by 

(x'=+\/+\ ^^+1) = (a;(a),p(a), w(a)) + (Ax, Ap, Ac.), 



(63) 



M 



k+i'- k+i 
fc+i _ P ^ 

2n 



(64) 
(65) 



where Ap = (Ay, Az) and Acj = (AA, A7). Denote P{a) = 



Y{a) 

Z{a) 
and D — P2 (q;)57^ 2 («). Then, the last 2 rows of (|63|) can be rewritten as 



n{a) = 



A(a) 







r(a) 



PAu + QAp = u{a)e - P{a)n{a)e. 
Now, we show that the correction step brings the iterate from M^i'i'O) back to NiiO). 



(66) 



Lemma 5.10 Let{x{a),p{a),oj{a)) e J^2{29) and {Ax, Ap, Auj) be defined as in (S3>- iei (x'=+\p'=+\ w'^+i) 
be updated by using [64\ ). Then, for 6 < 0.29 and sin(a) < sin(a), (x''+^,p''+^, w''+^) e N2(0). 

Proof: See Appendix A. 

Next, we show that the combined step (searching along the arc in A/2 (20) and moving back to N2{d)) 
will reduce the duality gap of the iterate, i.e., /i'^+^ < /i'^, if we select some appropriate 9 and a. We 
introduce the following two Lemmas before we prove this result. 



Lemma 5.11 Let {x{a),p{a),(jj{a)) ^ J\f2{29) and {Ax, Ap, Auj) be defined as in i6S\) . Then 

Ap'^Au: 9H1 + 29) So , . 

< — < —7-^ ;r7TT^M(a) := —^{a). 



2n 



n{l-29y 



(67) 
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Proof: See Appendix A. 

Lemma 5.12 Let{x{a),p{a),u}{a)) e Af2i29) and {Ax, Ap, Acj) be defined as in (S3)- Let {x''+'^ ,p''+^,u}''+^) 
be defined as in |6'^[ j. Then 

fe+i p'=+i^w'=+i ,,A 02(^^20)\ ,,/ So 



Proof: Using the fact that p{a)'^ Aoj + uj{a)'^ Ap — established in (IllOp in the proof of Lemma FS-lOi 
and Lemm.a 15.111 it is therefore straightforward to obtain 



^(")"-(") + ±Ap-A. ^ (M-) + A.r(.(.) + A.) ^ ^ ^^(1 + 2^) ^ 



This proves the lemma. | 

For linear programming, it is known |13 | 122 ) that /z = fi{a). This claim is not always true for the 
convex quadratic programming as is pointed out in Lemma 15.121 Therefore, some extra work is needed 
to make sure that the duality gap will be reduced in every iteration. 

Lemma 5.13 For 9 < 0.19, if 

sin(a) =: —=, (68) 

then p'^^^ < p,'^ . Moreover, for sin(Q;) = -y= — ^^^ , 

/-<,^(l-M|^). (69) 



Proof: See Appendix A. 

Remark 5.3 As we have seen in this section that starting with {x'^ , p^ , uj^) , the interior-point algorithm 
proceeds with finding {x{a),p{a),uj{a)) G JV2{29) and (x''+^,p''+\ i:j''+^) G JV2{9) such that p^+'^ < 
p . In view of the proofs of Lemmas \5.7\ \5.10\ and 15.-? 51 the positivity of {x{a),p{a),uj{a)) > and 
(a;'^+^,p''"+^,a;'^'+^) > relies on p{a) > which, according to Corollarv \5.1[ is achievable for any 9 and 
is given by a bound in terms of a. The proximity condition for {x{a),p{a),uj{a)) relies on the real positive 
root of q{sm{a)), denoted by sin(Q:), which is conservatively estimated in Lemma 1 5. 9\ under the condition 
that 9 < 0.22; the proximity condition for (a;'^+^,p'"'+^, aj'^+^) is established in Lemma \5.10\ under the 
condition that 9 < 0.29. Finally, duality gap reduction jJ^^^ < p^ is established in Lemma \5.13\ under the 
condition that 9 < 0.19. For all these results to hold, we just need to take the smallest bound 9 — 0.19. 
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We summarize all the results in this section as the following theorem. 

Theorem 5.2 LetO ^ 0.19 and{x^ ,p^ ,uj^) eAf2{0). Then, {x{a),p{a),u}{a)) eAf2{2e); {x''+\p''+^ ,uj''+^) G 



M2{9); andn^+^ </ {l 



0.0185 



Proof: From Corollary 15.11 and Lemma |5.9[ we can select sin(Q;) < min{sin(a),sin(Q:)}. Therefore, 
Lemma 15.71 holds, i.e., {x{a),p{a),ijj{a)) G J\f2{29). Since sin(a) < sin(a) and {x{a),p{a),uj{a)) G 
7V2(26i), Lemma [530] states (x'''+\/+\ w'^+i) G ^'2(6'). For 6 = 0.19 and sin(a) = ^, LemmaEH 
states fi''+^ < m'' (1 - ^^'^) ■ This finishes the proof. | 

Remark 5.4 It is worthwhile to point out that = 0.19 for the box constrained quadratic optimization 
problem is larger than the 9 = 0.148 for linearly constrained quadratic optimization problem. This makes 
the searching neighborhood larger and the following algorithm more efficient in computation than the 
algorithm in 



We present the proposed method as the following 

Algorithm 5.1 (Arc-search path- following) 

Data: H >0, c, n, 9 = 0.19, e > 0, initial point (a;°,p°,a;°) G ^2(9), and fi° = 2^^. 
for iteration k — 1,2, . . . 

Step 1: Solve the linear systems of equations \21^ and \2^) to get (x,p,uj) and (x, j>,w). 

Step 2: Let sm{a) — -y=. Update {x{a),p{a),uj{a)) and ^{a) by |57[ ) and I5g|) . 

Step 3: Solve ^ to get {Ax,Ap,Auj), update {x''+\ p''+\ uj''+^) and p^+^ by using (^ and ifg5)] . 

Step 4-' Set fc + 1 — ^ fc. Go back to Step 1. 

end (for) 

6 Convergence Analysis 

The first result in this section extends a result of linear programming (c.f. [20]) to convex quadratic 
programming subject to box constraints. 

Lemma 6.1 Suppose T" 7^ 0. Then for each i^ > 0, the set 

{{x,p,uj) \ {x,p,uj) e T, p'^uj<K} 
is bounded. 
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Proof: First, x is bounded because — e <^ x < e. Since x + y = e and — e < x < e, we have < y = 
e — X < 2e. Since x — z = — e, we have 0<z = x + e< 2e. Therefore, y and z are also bounded. Let 
(x, y, z, A, 7) be any fixed point in T°, and (x, y, z, A, 7) be any point in J- with y^'^A + z'^'^ < K . Then 

iI(S - x) + (A - A) - (7 - 7) = 0. 

Therefore 

{x - xfH{x -x) + {x- xf{X - A) - (x - xf{^ - 7) = 0, 

or equivalently 

{x - x)^{^ - 7) - (x - x)^(A - A) = (x - xfH{x - x) > 0. 

Using the relations x — e — ~y and x + e = z, we have 

((x + e) - (x + e))T(7 - 7) - ((x - e) - (x - e))T(A - A) > 0, 

or equivalently 

(z-zf(7-7) + (y-y)T(A-A)>0. 

This leads to 

z'^7 + z'^7 - z'^7 - z'^7 + y'^X + y^ X - y^X - y^X > 0, 



or in a compact form 



Sine (p, w) > is fixed, let 



rpi rp rp rpi 

P UJ + p LU ~ p U] ~ p UJ >{). 



^= min min{pj,a;J. 

2— I,--- ,n 



Then, using p^uj < K, 

p u! + K > £_e {p + uj) > max max{^pi,^a;i}, 

i— I.--- ,n 

i.e., for i G {1, • • • ,7i}. 



0<ft< -(i^+P^w), 0<uj,<-iK+p'^Ld). 



This proves the lemma. 



The following theorem is a direct result of Lemmas 16.11 \5A\ Theorem 15.21 KKT conditions. Theorem 
A.2 in [20] . 

Theorem 6.1 Suppose that Assumption 1 holds, then the sequence generated by Alaorithm \5.1\ converges 
to a set of accumulation points, and all these accumulation points are global optimal solutions of the 
convex quadratic programming subject to box constraints. 
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Let (x* ,p* ,uj*) be any solution of (IT71) . following the notation of |^, we denote index sets B, S, and 
T as 

B={je{l,...,2n}\p*^0}. (70) 

S = {je{l,...,2n}\uj*^0}. (71) 

T={je{l,...,2n}\p*=uj*=0}. (72) 

According to Goldman- Tucker theorem [3, for the linear programming, B O S = (d = T and B U S = 
{1, . . . ,2n}. A solution with this property is called strictly complementary. This property has been 
used in many papers to prove the locally super-linear convergence of interior-point algorithms in linear 
programming. However, it is pointed out in 8 that this partition does not hold for general quadratic 
programming problems. We will show that as long as a convex quadratic programming subject to box 
constraints has strictly complementary solution(s), an interior-point algorithm will generate a sequence 
to approach strict complementary solution(s). As a matter of fact, from Lemma |6. 11 we can extend the 
result of |20[ Lemma 5.13] to the case of convex quadratic programming subject to box constraints, and 
obtain the following lemma which is independent of any algorithm. 

Lemma 6.2 Let ^^ > 0, and p G (0,1). Assume that the convex QP jib]) has strictly complementary 
solution(s). Then for all points {x,p,uj) with {x,p,uj) € J-"°, piUJi > pfx, and /i < /i*^, there are constants 
M , Ci, and C2 such that 

llb,^)ll<^^, (73) 

< p, < fi/Ci {i eS), 0<uji< Ai/Ci {i e B). (74) 

Lo^ > C2P {i e 5), p. > C2P {i e B). (75) 

Proof: The first result ((75| follows immediately from Lemma [6.1 1 bv setting K ~ 2n/i". Let (x*,p*,a;*) 
be any strictly complementary solution. Since {x* ^p* ,uj*) and {x,p,Lo) are both feasible, we have 

(y - y*) = -(x - X*) = ~{z ~ z*), H{x - x*) + (A - A*) - (7 - 7*) = 0. 

Therefore, 

(y - y*fiX - A*) + {z- z*f{^ - 7*) = {x ~ x*fH{x - x*) > 0. (76) 
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Since {x* ,y* ,z* ,X* ,j*) = {x*,p*,uj*) is strictly complementary solution, T = 0, _p* = for i £ S, and 
u* =0 ior i e B. Since p^uj = 2nfi, {p*)^u* = 0, from ([71]), we have 

pT^ = y^X + zT^ + {{y*)^X* + {z*)^r) > y^A* + zJj* + {{y*)'^X + (z*)^^) = pT^* + c^V 
^:^ 2n^ > p^uj* +uj^p* = EiesP^^j* + EjgbP?^*- (77) 

Since each term in the summations is positive and bounded above by 2n/i, we have for any i £ S, ui* > 0, 
therefore. 

Denote D.d = {{p* , uj*)\uj* > 0} and ftp = {{p*,uj*)\p* > 0}, we have 

0<p.< '-^ ,. 

This leads to 

2n/x 



max Pi < 

iGS mm.es sup(p. ^^.^^^n < 

Similarly, 

2n/i 



maxoji < 
lee mmiessup(p. ,^.)gf2^p* 

Combining these 2 inequalities gives 

2n/i 
max{maxpi, maxwi} < 



ie5 ' ise min{min,e5sup(p._„.)gt2^ w*,minjg|3sup(p. ,^.)gf^^p*} Ci ' 

This proves (|74p. Finally, PiUJi > pfi, hence for any i € S, 

p^ p^I 
Wi > — > -—^ = C2P. 
Pi A^/Ci 



Similarly, for any i £ B, 



P,> — > -^TT = C2P- 

LUi p Cy 



Lemma W% leads to the following 

Theorem 6.2 Let {x'' , p'^ , uj'') G M2{6) he generated by Aloorithms \5.1[ Assume that the convex QP with 
box constraints has strictly complementary solution(s). Then every limit point of the sequence is a strictly 
complementary solution oj the convex quadratic programming with box constraints, i.e., 

^* > C2P {i € 5), p* > C2P (i € B). (78) 
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Proof: From Lemma [6.21 (jJ^^ijj^) is bounded, therefore there is at least one limit point {jf,Lo*). Since 
{p^,ujf) is in the neighborhood of the central path, i.e., pfw*^ > p/i'^ := (1 — 36')/i'^, 

w.f>C2P (teS), p't>C2P {i&B), 

every limit point will meet ([75)) due to the fact that C2P is a constant. | 

We now show that the complexity bound of Algorithm 15.11 is 0{^/n\og{l / e)) . We need the following 
theorem from 20 for this purpose. 

Theorem 6.3 Let e G (0, 1) be given. Suppose that an algorithm, for solving ( |J7| ) generates a sequence 
of iterations that satisfies 

//+!< A-AV/, fc = 0,l,2,..., (79) 

for some positive constants S and x- Suppose that the starting point {x*^ , p'^ , u'^) satisfies ji^ < 1/e. Then 
there exists an index K with 

K = 0{n>^\og{l/e)) 

such that 

fj,'' < e for Vfc > K. 



Combining Lemma 15.131 and Theorems 16.31 gives 
Theorem 6.4 The complexity of Alaorithm \5.1\ is bounded by 0{^/n\og{l/e)). 

7 Implementation Issues 

Algorithm lS.ll is presented in a form that is convenient for the convergence analysis. Some implementation 
details that make the algorithm effective and efficient are discussed in this section. 
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7.1 Termination criterion 

Algorithm 15.11 needs a termination criterion in real implementation. One can use 





m'' < e 


rxW 


= \\Hx^ + X^ --f^ + c\\ <e 




ry = a;*" + u'' ~ e\\ < £ 




\\rz\\ = \\x'' - z'' + e\\ <e 




\\rt\\ = WP'^n'^e- ^J,e\\ < e 




ip\uj'')>0 



An alternate criterion is similar to the one used in linprog [26j 

llryll + llr^ll , \\rx\\ , / 



(80a) 
(80b) 
(80c) 
(80d) 
(80e) 
(80f) 



< e. (81) 



2n max{l,||c||} max{l, ||a;'''^iJa;'= + c^.t'^II} 

7.2 Initial (x°,AO,s°) €^(0) 

For feasible interior-point algorithms, an important prerequisite is to start with a feasible interior point. 
While finding an initial feasible point may not be a simple and trivial task for even linear programming 
with equality constraints [5], for quadratic programming subject to box constraints, finding the initial 
point is not an issue. We show that the following initial point (x",?/*^, z", A'',7*') is an interior point, 
moreover (a;°, y*^, z", A",7'^) eJ^2{S)- 

a;0 = 0, 2/° = z" = e > 0, (82a) 

A°=4(l + ||c|p)-|>0, (82b) 

7° = 4(l + ||c|p) + |>0. (82c) 

It is easy to see that this selected point meets pO|. Therefore, we will show that it meets p4)) . Since 

^0^Ski^Ltii.SkWi±l!ifl).4(l + |lcf,, (83) 

2n 2n 



we have, for 9 — 0.19, 

^ n n 



/ o ^0 - ^Oe 



\cr 
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7.3 Step size 

Directly using sin(Q:) = -y= in Algorithm 15.11 provides an effective formula to prove the polynomiality. 
However, this choice of sin(Q;) is too conservative in practice because this search step in Af2{'29) is too 
small and the speed of duality gap reduction is slow. A better choice of sin(a) should have a larger 
step in every iteration so that the polynomiality is reserved and fast convergence is achieved. In view of 
Remark l5.31 conditions that restrict step size are positivity conditions, proximity conditions, and duality 
reduction condition. We examine how to enlarge the step size under these restrictions. 

First, from (I104|l and (IllSp . /i(a) > is required for positivity conditions {p{a),uj{a)) > and 



iP' 



fc+l ,.k+l 



) > to hold. Since sin(a) estimated in Corollary 15.11 is conservative, we find a better a 



directly from (|54p . 

M(a) > m(1 - sin(a)) - -—(p^uj) ( sin'*(a) + sin^(a) ) := /(sinfa)) = a, (84) 

where cr > is a small number, and /(sin(Q!)) is a monotonic decreasing function of sin(a) with /(sin(O)) = 



1 and /(sin(|)) < 0. Therefore, 



has a unique positive real solution for a £ [0, -j] Since (|84p is a 



quartic function of sin(Q:), the cost of finding the smallest positive solution is negligible |10) . 

Second, for 9 < 0.19, from (|112|) . the proximity condition for (a;''+\ y''+^, 2;''+^, A''+^,7''+^) holds 
without further restriction. The proximity condition ()103|) is met for sin(Q!) e [0, sin(a)], where sin(a) is 
the smallest positive solution of ([5^ and it is estimated very conservatively in Lemma [5.91 An efficient 
implementation should use sin(a), the smallest positive solution of (1561) . Actually, there exist a d which 
is normally larger than a such that the proximity condition (I103P is met for sin(a) € [0, sin(d)]. Let 

60 = -9^J. < 0, 



63 = 

64 = 



61 = 6l/i > 0, 



1 



po LJ + di o p (p^i^ + Lo'^p)e -\ ip^t^ + P^f^) , 

2n n 



po uj — UJ o p - 



2n 



{p LJ ~ U! p)e (p LJ ~ p w) , 



and 



p{a) :— 64(1 — cos(a))^ + 63 sin(a)(l — cos(a)) + 61 sin(Q;) + 60. (85) 

Applying the second inequality of p5)) to ^ {"P^i^ + p^ili^ sin(a)(l — cos(a)), we can easily show that 

p{a) < q{a), 
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where q{a) is defined in (1551) . Therefore, the smallest positive solution a oip{a) is larger than the smallest 
positive solution a ofq{a). We will show that for sin(a) G [0, sin(Q;)], the proximity condition (|103p holds. 
Since for sin(Q;) € [0,sin(a)], p{a) < 0, we have 

pouj — Loop (p w — uj p)e (1 — cos(a)) + poij + iljop (p a) + w p)e sin(Q;)(l — cos(a)) 

2n 2n 

< (29) ( — (jF^-p^Lu) (l-cos(a)f - — (p^u + p^Lu) sm(a)(l - cos( a))] - 61^(1 - sin(a)). (86) 
\2n 2n J 

Substituting this inequality into (I102p gives 

p{a) o uj[a) — ^{a)e 

< 26l(^(l - sin(a)) + ^ (i'^{j - A) - ±^(7 - A)) (1 - cos(a))2 

- ^ (i^il - A) + ^"^(7 - X)) sin(a)(l - cos(a))] = 26l^(a). (87) 

This is the proximity condition for {x(a),y{a), z{a), X{a)^j{a)). Denote 60 = bo, hi = 61, 

63 if 63 > 0. . f 64 if 64 > 0, 

if 63 < 0, [ if 64 < 0, 

and 

p{a.) :— 64(1 — cos(a))^ + 63 sin(a)(l — cos(a)) + 61 sin(Q;) + 60. (88) 

Since p(a) > p{a), the smallest positive solution a of p(q;) is smaller than smallest positive solution a of 
p{a). To estimate the smallest solution of a, by noticing that p{a) is a monotonic increasing function of 
a and p(0) = — 0/i < 0, we can simply use the bisection method. The computational cost is impendent 
of the problem size n and is negligible. Since both estimated step sizes a and a guarantee the proximity 
condition for (x{a),y{a),z{a),\{a),^{a)) to hold, we select a — max{(i,a} > a which guarantees the 
polynomiality claim to hold. 

Third, from (|121ap and Lemma [531 we have 



M 



fc+i 



< 



^^ 



1 



^1 + 20) 



- 1 



n(l- 261)2 
For ^^^^ < ^^ to hold, we need 

'2(1 + 26*) f , 6l2(l + 26l) 



6(2(1 + 261) 
n(l-26')2 



02(l + 20)\ pT^ 



sin(a) + 1 + 



sin(a) + 1 , ,_ ^^,„ , „ 
= (l + 20)\ p^Q I . 



(sin^(a) + sin"* (a)) 



(sin2(a)+sin'*(a)) < 0. 



n(l- 261)2 y n(l-26')2y ' ' \ n{l - 29^ J 2n^i 
For the sake of convenience in convergence analysis, a conservative estimate is used in Lemma 15.131 



For efficient implementation, the following solution should be adopted. Denote u = 
V — 2-^ > 0, z = sin(a) £ [0, 1], and 

F{z) = (1 + u)vz'^ + (1 + u)vz^ - (1 + u)z + u. 



1(1-20) 



> 0, 
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For z € [0, 1] and u < i, i^'(2;) = (1 + u)(Avz^ + 2vz — 1) < 0, therefore, the upper bound of the duahty 
gap is a nionotonic decreasing function of sin(a) for a G [0, ■!]. The larger a is, the smaher the upper 
bound of the duahty gap wih be. For v > ^, to minimize the upper bound of the duahty gap, we can find 
the sohition of F'{z) = 0. It is easy to check from discriminator 15 that the cubic polynomial F'{z) has 
only one real solution which is given by (see Lemma 12.51) 



sin(a) 



nfi 



\ 4p^ 



> 2 /I \ 3 



4pTaJ 



nfi 



6/ \ 4pTw V \^P'^^ 



I / \ 2 /I \ 3 



Since F"(sin(Q!) = (1 + u)(12i;sin^(a) + 2v) > 0, at sin(a) e [0, 1), the upper bound of the duality gap 
is minimized. Therefore, we can define 



a = < 



if f^ <i 

2nfi — D 



(89) 






( Jit 



ye) ^ y Ap'^ui 






It is worthwhile to note that for a < a, i^'(sin(a)) < 0, i.e., _F'(sin(Q;)) is a monotonic decreasing function 
of a e [0, a]. 

We summarize the step size selection process as a simple algorithm as follows. 

Algorithm 7.1 (Step Size Selection) 

Data: a > 0. 

Step 1: Find the positive real solution of |<g.^[ ) to get sin(Q;) 

Step 2: Find the smallest positive real solution of i88\) to get sin(d), the smallest positive real solution of 

i56\) to get sin(a), and set sin(a) = max{sin(a),sin(d)}. 

Step 3: Calculate a given by i89\) 

Step 4-' The step size is obtained as sin(a) = min{sin(Q;), sin(Q!), sin(Q!)}. 



7.4 The practical implementation 

Therefore, Algorithm 15 . II can be implemented as follows. 



Algorithm 7.2 (Arc-search path- following) 

Data: H >0, c, n, ^ 0.19, e > ct > 0. 

Step 0: Find initial point (a;",p°,a;") € N2iO) using \82^) . k using \81\) . and ijP using 
while K > e 
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Step 1: Compute {x,p,uj) and {x,p^ui) using {21^ and \2^) . 

Step 2: Select s\n.(a) using Algorithm \7. 1\ Update {x{a),p{a),uj{a)) and ii{a) using ( [37P and \5l^. 

Step 3: Compute {Ax,Ap,Au!) using if^] . update (a;'^+\p'^+^, w'^+^) and ^x^^^ using (5^ and if^] . 

Step 4-' Computer k using i81\) . 

Step 5: Set fc + 1 — >■ fc. Go back to Step 1. 

end (while) 

Remark 7.1 The condition fi > a guarantees that the equation ^84^ has a positive solution before ter- 
minate criterion is met. 



8 A design example 

In this section, we will use the design example of [1] to demonstrate the effectiveness and efficiency of 
the proposed algorithm. The linear time-invariant system under consideration is given by 



Xs+l 



I 


h 


X, + 





-/l 


1 




h 



Us=Axs+Bus, s = 0,...,iV- I, 



with the initial state given by xt — [15, 5]"''. The control constraints are 

-1<U5<I, s==0,...,A^- I. 

The problem is to minimize 

1 h ^~^ 

J = min -x^PxAT + TT V [xfc Qxfc + Rul,] 



where the matrices P, Q, and scalar i? are given by 



fc=0 



P = Q = 



2 
1 



R = 6. 



This problem arises from discretization of the continuous-time problem of minimizing 










subject to 



x(t) 






1 


x(t) + 





-1 







1 



[x(i)'^Qx(i) + Ru{tf] dt 



u(t) = y4x(t) + Bu{t), 



(90) 



(91) 



(92) 
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T_ 

N- 



and 

l<u(t)<l, te[0,T], 

where the interval [0,r] is discretized into N intervals of length h 

In our implementation of Algorithm I7.2[ e — 10^* and a = 10^^° are selected. For the simple 
design example, T = 50 and A^ — 500 are used. The original quadratic optimization problem has 
Nr + Nm — 1500 variables, Nr — 1000 equality constraints, and 2Nm — 1000 inequality constraints. 
The reduced problem has Nm = 500 variables, no equality constraints, and 2Nm — 1000 inequality 
constraints, a significantly simpler problem. The advantage will be even more significant if the state 
dimension r is significantly larger than the control dimension m. After 27 iterations, the algorithm 
converges. Using the optimal control inputs, we can calculate the state space response from (j90p . The 
control inputs and state space response are displayed in Figure 1. 



Optimal control and system state response 




20 30 

t (seconds) 



Figure 1: Optimal control with saturation constraint. 



9 Conclusions 



This paper proposes an arc-search interior-point algorithm for convex quadratic programming subject 
to box constraints that searches the optimizers along ellipses that approximate the central path. The 
saturation constrained LQR design is one such problem. The algorithm is proved to be polynomial 
with the complexity bound 0(Y^log(l/e)). A constrained LQR design example from |4l is provided to 
demonstrate how the algorithm works. Preliminary test on this simple design problem shows that the 
proposed algorithm is promising. A MATLAB M-file implementation of Algorithm 17.21 is available from 
the author. 
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10 Appendix A: Proofs of Technical Lemmas 



Proof of Lemma \5.2[ 

From (j30p . we have i"^ {'j — A) = z'^j + y'^ X = "p^uj, x"^ {'y — A) = z'^j + y'^X = p^cj, ^"^(7 ~ -^) = P^^i 

and x"^ {j — A) = p^ui. Pre- multiplying x"^ and x"^ to (|29p gives 

x^(7-A) = i'^iJi, 

a; (7 — A) = X Hit, 

^^(7 - A) = i'^iJi == x^Hi = i'^(7 - A). 

((4T|) and (|42)) follow from the first two equations and the fact that H is positive definite. The last equation 
gives (gSl). Using (gH), dH]), and ^ gives 

(a;(l — cos(a)) + x sin(a)) H{x{l — cos(a)) + x sin(a)) 
= {x'^Hx){l - cos(a))2 + 2{x^Hx) sin(a)(l - cos(a)) + (x^Hx) sin^(a) 
= {x'^Hx){l - cos(a))2 + (x^Hi) sm^{a) + {i^{j - A) + ±'^(7 - A)) sin(a)(l - cos(a)) > 0, 

which is the first inequality of dH]). Using (gT]), (gl]), and (021) also gives 

{x{l — cos(a)) — a;sin(a))'^iJ(i;(l — cos(a)) — isin(a)) 
= {x'^Hx){l - cos(a))^ - 2{x^Hx) sin(a)(l - cos(a)) + [x^Hx) siiv^{a) 
= {x'^Hx){l - cos{a)f + {x^Hi) iiin^{a) - (^'^(7 - A) + x^ {-/ - A)) sin(a)(l - cos(a)) > 0, 

which is the second inequality of (j44p . Replacing ±(1 — cos(a)) and a;sin(Q;) by i:sin(a) and i(l — cos(q;)), 
and following the same method, we can obtain (|^5|). | 



Proof of Lemma \5.3[ 

From the last two rows of ([?7)) or equivalently (PT|) . we have 

Ay + YX = AYe 
Pi + Z7 = TZe. 

Prc-multiplying Y^^A^^ on both sides of the first equality gives 

Y^^A^ij + F2 A^2 A = F2 A2 e. 
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Pre- multiplying Z 2 r 2 on both sides of the second equality gives 



Let u 



Y 2 Aay 



Z-^V^z 



, w 





, and w = 





(93) 



use (15(1)) and Lemma 15. 2[ we have 



u"^w = y'^A + i'^7 = i"^(7 — A) > 0. Using Lemma [^751 and ([25]) . we have 



\\A 



E 



2/f^» , ^hi\ , V- M?y« . 7i^^« 



+E 



< ^ (yiAi + Zi7i) = ^Pit^i = 2n/x. 



y* Zi J ^ y Aj 7i 

Since Pi > and w^ > 0, dividing both sides of the inequality by miuj p^Wi and using (P5)) gives 



• 9 -2 

y' , -4 



gd^tj^gd^l 



p 2 



< 



2nfj, 



< 



2n 



miiij piUJi 



1 



(94) 



This proves (|47|). Combining (j47|) and Lemma 12711 vields 



< 



(i-e) 



This leads to, 



< 



Therefore, using (j25p and Cauchy-Schwarz inequality yields 



E^<\&1<^1 + , 



,^^^<(i + ^)fMVm)<(i + ^) 



< 



1 + 1 



i"-' 



(95) 



(96) 



which is the second inequality of (H^ . From Lemma [5.21 p^uj = x'^{j ~ X) — x^Hx > 0, we have the 
first inequality of (l49t . | 



Proof of Lemma \5.4\ 

Similar to the proof of Lemma [5.31 from (|5^ . we have 



Ajj + YX 



-2 (y o X\ 



and 



y-^A^y + Y^A-^X = -2Y-iA-i (y o aV 
Tz + Z7 = —2 (i o 7) 
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Let u 



Y 2 A2 j/ 



z-^riz 



Y2A-2X 



and w 



-2z-ir-i (207) 



Z2r-2j 

5.21 u^v = y'^X + z^^ = x^{'y — X) > 0. Using Lemma [2.31 we have 



, using pop and Lemma 



h\ 



y^ (ylh. + ^'i^' 



< 



Vi 



-Tf-^K-^ 






E 

|2 



X2„. A-,2 



liZ; 



yoAj + -2Z"5r-2(io7) 

2» li I ' 



Dividing both sides of the inequahty by /i and using (j25p gives 



"-''IE>„, 






E 



A? 
A? 



< 4(14 
in view of Lemma 15.31 this leads to 



l-l 



UJ 
2 \2 






2 A 

< 4— 



p 





UJ 


p 




LO 



2 .1 

< 4— 



2 4.{l + 6)r? 

- {I -Of 



(97) 



This proves ([50]) . Combining ([50]) and Lemma [^TT] yields 



2 /2(1 



{I -Of 



Using ((25|) and Cauchy-Schwarz inequality yields 



p^ < IpII^I < ^ 



< (1 + 1 



^)(fi^<> 



< 



2n2(l 



(i-ey- 



/I /i maxi piUJi 

which is the second inequality of ([5^ . Using ([50)) and LemmajS^l we havep"'"w = y'^A+z"'"7 — ic^ {^—X) — 
x^ Hx > 0. This proves the first inequality of ([5^ . Finally, using ([^. Cauchy-Schwarz inequality, (|Tf| . 
and ([501) yields 

,|T|-M l_ilTl-M /l_-l\ T 



Ip^^^i < bri^i < /^ 



A* 



<(1 



<fl- 



) '^'"'"' <(i + g)('M) (^ 



2n 



4(l + 6i)n^ 



2\ 2 



< 



(2n(l + 0)): 



1-9 J V (1 + ^)^ / " (1-61)2 • 
This proves the first inequality of ([53]) . Replacing phj p and a) by w, then using the same reasoning, we 
can prove the second inequality of (f53| . | 
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Proof of Lemma \5.5\ 

Using dMl), dMl), dSD), and ^, we have 

^yiP' ^ y"''sin(a) + j/''"(l — cos(a)) j (A — Asin(a) + A(l — cos(a)) 1 
=y"^A — y'^Asin(a) + y"^A(l — cos(a)) 

— j/'^Asin(a) + y^ \siv?'{a) — y^ \shi{a){\ — cos(q;)) 

+ ij^X{l ~ cos(a)) - y^\s\n{a){l - cos(a)) + y^\{l - cos(a))^ 
=y^X - {y^\ + X^y) sin(a) + (y'^A + X^y){l - cos(a)) 

— {y'^X + X^y)sin{a){l - cos(a)) + 2;'^Asin^(Q;) + y^ X{1 - cos(a))^ 
=y^A(l - sin(a)) - 2y^ X{1 - cos(a)) 

— (y'^A + X^y) sin(a)(l — cos(a)) 

+ y^A(l - cos2(a)) + i/'^A(l - cos{a))^ 
=y^X{l ~ sin(a)) + {fx - ?/'^A)(l - co%{a)f - {ifx + X^y) sin(a)(l - cos(a)). (98) 

Using ([55)) . ([57]) . ((5T1) . (|5^ . and a similar derivation of ([Ml) , we have 

z'^(a)7(Q;) = z'^7(l — sin(Q;)) + (z'^7 — z'^"/){l — cos(a))^ — {z'^j + '^'^ z) sin(a)(l — cos(a)). (99) 

Combining ([98)1 and (|99)) gives 

2nii{a) — p^{a)uj{a) 
=y (Q;)A(a) + z {a)j{a) 
= {y^X + z^j){l - sin(a)) + (y'^A + z^j - y'^ X - z'^'j){l - cos{a)f 

— [ip'X + z^'^ + y^X + 'z}^^ sin(a)(l — cos(a)) 

= (y'^A + z^i){\ - sin(a)) + (^'^(7 - A) - ±'^(7 - A))(l - cos(a))2 use ^ 

- (±'^(7 - A) + x^(j - A)) sin(a)(l - cos(a)) (100) 
<(y'^A + z'^7) (1 - sin(a)) + {x^ H'x - x^ Hx){\ - cos(a))^ use (gi]) in Lemma ES 

+ x^Hx{\ - cos(a))^ + x^Hx sin^(a) 
= (y'^A + z'^7) (1 - sin(a)) + x^ llx{\ - cos(a))^ + x^Uxsv[?{a). 

Dividing the both side by 2n proves the second inequahty of the lemma. Combining (|100p and (|45t proves 
the first inequality of the lemma. | 
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Proof of Lemma \5.6\ 

From the second inequality of (15^ . we have 



a) — /i < /^ sin(Q!J — 1 H sin(Q;) H sni (a) . 

2nfi 2nfi 



Clearly, if %f^ < |, for any a G [0, -1], the function 



/(a) := -1 + — sin(a) + — siT?{a) < 0, 



2nfj, 



2n/i 



and ii{a) < /i. If 2_2£ > 1 using Lemma [2.51 the function / has one real solution sin(Q;) € (0, 1). The 



2nfj. ^ 2' 



solution is given as 



sin(ci) 
This proves the Lemma 



nfi 






1 



nfj, 
x^Hx 






Proof of Lemma \5.7\ 

Since sin(Q;) is the only positive real solution of ()56p in [0, 1] and q{0) < 0, substituting ao, ai, 02, 03 and 

04 into (|56p . we have, for all sin(Q;) < sin((5), 



pooj — ojop— 7— (p LJ — a; p)e ) sin (a) + pooj + cjoji — -— (p 6j + lj p)e ] sin (a) 
\ 2n J \ 2n J 



< 



20^^'^^„:..4/„,^ fnnP^^\„^^2 



sm^{a) - 26^—— ) sin^(a) + 6'/i(l - sin(a)). 



(101) 
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Using dSHl), dSni), (ED), (EH), dSHl), Lemma O pi?T|) . and the first inequality of dMl), we have 

p{a) o uj[a) — /i(a)e 

ip — psin(Q;) + p(l — cos(q!)) ) o I cj — cj sin(Q;) + d}(l — cos(q;)) ) — /i(a)e 

(po u) — /ie)(l — sin(a)) + [pouj — pouj (p uj — p uj)e ] (1 — cos(a)) 

\ 2n J 

— [poij + ujop (p^u) + p^uj)e I sin(a)(l — cosfa)) 

\ 2n J 

<(1 -sin(a)) 



p o u! — fie 



(po oj — po oj (p^uj — p^u}))e (1 — cos(a))^ 

2n 



(p o LJ + ij o p — ——(p UJ + p uj)e sin(a)(l — cosfa)) 
2n 

<0/i(l — sin(a)) + {p o LJ ^ p o iJj (p s—p w))e sin (a) + 03 sin (a) 

2n 

•T • 

<26'/x(l - sin(a)) - (26?-^) (sin^ia) + sin^^a)) 
V 2n / 

<2e ( /i(l - sin(a)) - (^-y^) ((1 - cos(a))^ + sm^{a)) 
<29n{a). 



(102) 



(103) 



Hence, the point {x{a),p{a),uj{a)) satisfies the proximity condition for Af2{29). To check the positivity 
condition {p{a),uj{a)) > 0, note that the initial condition (p, w) > 0. It follows from (I103P and Corollary 
S.ll that. for sin(a) < sin(a) and 6 < 0.5, 



p^{a)uj,{a)> {l-20)p{a) > 0. 



(104) 



Therefore, we cannot have Pi{a) = or Wi(a) — for any index i when a £ [0,sin ^(a)]. This proves 
(p(a),L.(a))>0. I 



Proof of Lemma \5.8[ 
Since 



2n / ■ \ 2 



E 



2n / . s 2 

UJ, 



E 
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From Lemma 15.31 and (P5|) , we have 



> 



> 



> 



p 2 ,; 



E 

4=1 

2n 






2n / . X 2\ / 2n / . \ 2\ 



\i=l 



E £ E 



^j=i 



p to 

— o — 
p UJ 






{l + ef^l^ 



p O UJ 



I.e., 



This proves (|59)) . Using 



p O CJ 



< 



^si;,) . 



1- 



-n/i 



^E s 



i=l ^-^'' 1=1 

and Lemma [S^ then following the same procedure, it is easy to verify (pO)l . From (|T7|) and ([SO]) , we have 



2n \ (A{l + e)n^ 



il-0)J\ (1 



> 



p 2 



W 



P 



> 



\p ||2|| cj 



Ip II II W 
/ 2n 



2 ||p||2 II LJ ||2 



pll IICJ 
/ 2n 

£(£)) fe 



2\ / 2n / . \ 2N 



2n / . X 2N 



Vi— 1 / \«— 

2n / .. . \ 2 2n / . .. x 2 



E £ E 



2n / .. s 2^ 

UJi 



i=l 

2n 

> 



PiMH 

PiUJi 
PiUJ 



E 



Pi^i 



Sv(i + ^')J +lJl(i 



i=l 

2 2n 



Pil^i 



PiWj 



1 



(1+0)2^ 



2, ,2 



6)^i 

2 



p O Uj\\ + \\p O UJ 



I.e., 



This proves the lemma. 



PO UJ 



P O UJ 



2 (2n)3(l + 0)^,2 



< 



(1- 



-M 



(105) 



Proof of Lemma \5.9[ ' 

First notice that q(sin(Q!)) is a monotonic increasing function of sin(a) for a G [0, ^] and (7(sin(0)) < 0, 



35 



therefore, we need only to show that q{-y=) < ior 6 < 0.22. Using Lemma [321 we have 

1 



p O LJ + UJ O p (p LJ + (Jj p)e 

2n 



p o CO — uj o p (p ui — u! p)e 



< 



< 



p O OJ 



po UJ 



In view of Lemmas 
q(sin(a)) < 



and l5.4[ from ([55|. we have, for a € [0, ^] 



po (jj 



UJ o p 



.»^).,i„V) + ( 



p O UJ 



LJ O p 



UJ O p 



UJ O p 



sin^(a) 



•T • 

+ 26*^^ sin^ (a) + 9fi sm(a) ~ 9fi 
2n 



<m( 



2(1 + 0)2 ^ nil + 9) 9(1 + 9) 



+ 



(1-0)3 

9(1 + 9) . 2 

(1-0) 



(1-0) (1 

sin'^(a) + 0sin(a) — 



sin^(a) + 472 ;^^^;' ni sin3(a) 
(1 - 0)^ 



Since n> 1 and > 0, substituting sin(a) — -^ gives 



2(1 + g)^ 2 n(l + 0) 0(l + 0) \ 0^ , /:^ (l + 0)t»J 03 
(1 - 0)3 '^ (1-9) (1-9) J n^ (1 - 0)2 



712 



fl 



(1-0) n 

f^ 
' ^V (1-0)3 ' n(l-0) ' (1-0)^2 

4^202(1 + 0)1 02(1 + 0) 



203(1 + 0)2 03(1 + 0) 04(1 + 0) 



M^i 



1-0)2 n(l-0) V^ 

203(1+0)2 03(1+0) 04(1+1 



(1-0^3 

4^/202(1 



(1-0) (1 

32(1+0) 



-^fB^--')-«^)- 



(106) 



(1-0)2 ' (1- 

Since p{9) is monotonia increasing function of e [0, 1), p{0) < 0, and it is easy to verify that p(0.22) < 0, 
this proves the lemma. | 



Proof of Lemma \5.1(A ' 
Using Lemma |2.6[ we have 



0< 



1 2 

Apo Auj~ -—(Ap^Auj)e 
2n 



< ||Apo Aa;||2. 

_i 
Pre-multiplying (P{a)i}{a)] on the both sides of (jHS]) yields 

DAuj + D-^Ap = (P{a)n{a)] ' U{a)e - Pia)n{a)e] . 



(107) 
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Let u — ZJAo;, v = D ^Ap, from (|63p . we have 

u^v = Ap^Auj = Ay^AX + Az'^A'^ = Ax'^(A7 - AA) = Ax^HAx > 0. 
Use Lemma [2.41 and the assumption of {x{a),p{a),uj{a)) e Af2{2d), we have 



Apo Aw 



— \\u o v\\ < 2 2 

2n 



(P(a)f}(a)) ' (fJ-{a)e - P{a)n{a)e\ 



2 v^ (M(a)-Pi(a)w,(a))" 



2-^E 



i=l 



Pi(a)wi(a) 



< ^-3 ||^(a)e-p(a)ow(a)|p 
~ miiii pi{a)uji{a) 



(l-26l)^(a) (1-26I)' 

Define (p'=+i(i),a;'^'+i(t)) = (p(a),w(a)) +i(Ap, Aw). From §Q and (gOl), we have 

2)1 

p(a)'^Aa; + w(a)'^Ap = 2n^ — 2^Pi(a)'^i(a) = 0. 



(108) 



(109) 



(110) 



Therefore, 



.,1,, (p{a)+tAp] L(a)+tAw) p(a)T^(a) + i2^ T^^ ^Ap^Aw 



(111) 



2n 2n 2n 

Since Ap'^Aw = Ax'^HAx > 0, we conclude that fi''+^(t) > ^(a). Using ([TTT|) . (|66l) . (fTOTj) . and (fT09| . 
we have 



/+^(i)ow'^-+^(i)-/-+^(t)e 



(p(a) + tAp) o (uj(a) + tAuj) - fi(a)e (Ap'^Aw) e 

2n 

^2 

p{a) o uj(a) + t(uj{a) o Ap + p(a) o Aw) + t^Ap o Aw — iJ.{a)e (Ap Aw) e 

2n ^ ^ 

^2 

p(a) o w(a) + t(ii(a)e — p(a) o w(q:)) + t^Ap o Aw — /i(Q;)e (Ap'^Aw) e 

2n ^ ' 

(1 - i) (p(a) o w(a) - A*(a)e) + ^^ ( Ap o Aw - — (Ap'^Aw) e j 



< {l-t)(26)pi{a)+t^ 



ha2 



220 



(1 - 20) 



/x(a) 



< (l-f)(26i)+t 



^i5^)M-':^/(M)/«. 



(112) 



Therefore, taking t = \ gives 



p^+^ OljJ^+^ — ^^^+16 



< 7f!|^M''+^- It is easy to see that, for 9 < 0.29, 



ifl2 



2*6* 



= 0.2832 < e. 
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For e < 0.29 and t € [0, 1], noticing < f{t,9) < /(i,0.29) < 0.58(1 - t) + 0.2832^^ < 1, and using 
Corollary 15. 11 we have, for an additional condition sin(a) < sin~^(a), 



= (l-/(i,0))(^Ma) + ^ApTAc. 

>{l-f{t,e))fi{a) 

>0, (113) 

Therefore, {p''+\t) , uj''+^ (t)) > for t e [0, 1], i.e., (p'^+i, w'=+i) > 0. This finishes the proof. | 



Proof of Lemma \5.11l ' 

The first inequality of ([57)1 follows from (|108p . Pre-multiplying both sides of (|551) by P^2 (Q;)fi^2 (q-) 

gives 

P-^ (a)n^a)Ap + P^a)n-^a)Auj ^ p-^a)n-^a)(n{a)e - P{a)n{a)e\ . 

Let u — P^2 (a)r22 (q-)Ap and v ~ Ps (0^)57^2 (q-) Aw, and w = P^2 (q;)J7-2 (q-)( //(a)e — P{a)Vt{a)e\, 
from (|108p . we have v?-v = Ap^'^Aw > 0. Using Lemma [2.31 and the assumption of (a;(a),p(a),a;(a)) € 
J\f2{29), we have 



\uP + \\ve 



f{Ap,)^w,{a) , (Aw,)2p,(a) 



<ii-f = E 



_ V, 

fit V P»(a) 

{p,{a) - pi{a)uji{a))'^ 
Pi{a)uji{a) 



0Ji{a) 



< 



< 



mmipi{a)uji{a) 
(20)V(«) _(20)V(a) 



{l-2e)fi{a) (1-26*) 



Dividing both sides by p{a) and using pi(Q;)a;i(a) > ^{a){l — 26) yields 

2" // A N2 ,-A, , \2^ 



< 



:(l-26l) 

. {2ef 
■{i-2ey 



{Ap,y , (Ac.,)^ 



Pi (a) w2(a) 



Ap 

p(a) 


2 

+ 


Acj 
w(a) 



(114) 



(115) 
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Ap 

p{a) 


2 

+ 


Aw 
uj{a) 



Invoking Lemma |2.1[ we have 



This gives 



Ap 



< 



29 



1-26 



Aw 



p{a) w(a) 

Ap 



2 1 

< 



261 



4 V 1 - 261 



Aw 



< 



202 



p{a) w(a) - (l-26')2' 
Using Cauchy-Schwarz inequality, we have 

(Ap)T(Aw) 



Therefore, 



This proves the lemma. 



< 



E 



|Api||Awj 



M(a) 



<(l + 20)^ 
=(1 + 20) 



|Ak| |Aw,| 
'—^Pi{a) uJi{a) 
Ap |T Aw 



p(a) I ijj{a) 



<(l + 26l) 



Ap 



p(q:) w(a) 



Aw 



< 



26(2(1 + 261) 



(1-20)2 ■ 

(Ap)T(Aw) 02(^^20) 
2n - ?i(l - 20)2 



fj.{a). 



(116) 



(117) 



(118) 



(119) 

(120) 
I 



Proof of Lemma \5.13[ 

Using Lemmas 15.121 15. 5[ 12. 2[ 15.21 15. 3[ and 15. 4[ and noticing p'^iv > and p^oj > 0, we have 



n(l-20)2 



<^'= (1 - sin(a) + ^ sin4(a) + 
\ 2n/i 



•T ■■ 
p W 

2n^ 


+ 


■ T •• 

w p 
2n^ 



sin^(a) 1 + 



^^o 



fe / . . , , n(l + 0)2 . 4, , 2(2n)^(l + 0)i . 3, ,\ / <5o 
</i'' 1 - sm(a) + -^ -^ sm^(a) + ^ . ^ ^. „ ^ sm^(a) 1 + — 



(1- 



(1-0)^ 



(121a) 



(121b) 
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Substituting sin(a) 



,fc+i ^,,fe 



<A^ 1 - ^ 



-^ into (jl21bp gives 



./■ 1 



n (1-0)3 n2 (l-6')2 



n 



/Vi n(l-6l)3 



n(l-6l)2 






.^Mi__+- 



n n n(l — 0) 



3 I 
77,2 n 



n(l-6l)2 



6l4(l + 6l)2 2i6l3(l + 6')^ 



=/■ 1 



So 613(1 + 61)2 2502(1 + 6l)t 



V^9 V^(l - 



V^(l - 0)2 



n(l-0)3 7i(l-0)2 

3(1 + 61)2 2502(1 + 61)1 



1- 



01(1-0)3 V^(l-0)2 



Since 



03(l+6>)2 2102(1 + 61)5 ^ 03(1 + 61)2 2302(1 + 61)^ 



fiO), 



V^(l-0)3 v^(l-0)2 - (1-0)3 (1-0)2 

where /(0) is a monotonic decreasing function of 0, and for < 0.37, /(0) > 0. Therefore, for < 0.37, 



,fc+i 



<A^M 1 - ^ 



1- 



(5o 03(1 + 61)2 2502(1 + 6')i 



=/ 1- 



Since 



0(1 + 20) 03(1 



V^0 y/^{l - 0)3 7^(1 - 6')2 
0(1 + 20) 03(l + e')2 256*2(1 + e')5 

7^(r^20)^ " 7^0"^ " ^^a~^ 



2i02(l + 61)1 ^0(1 + 20) 03(1 + 0)2 2502(1 + 6')i 



(122) 



5(0), 



V^(l-20)2 ^{1-9)3 y/Ti{l-9)^ - (1-20)2 (1-61)3 (1-0)2 

where g{9) is a monotonic decreasing function of 0, and for < 0.19, g{9) > 0.0976 > 0. For = 0.19, 
05(0) > 0.0185 and 



fc+i ^ k f. 0.0185\ 
This proves ^^. | 
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